Soak, endurance, or reliability testing | Analyzing test results and solving performance problems
23.6 High Availability testing
In Chapter 20, Introduction to performance testing, we introduced the concept of High Availability testing in WebSphere Commerce performance testing. Generally, before starting the High Availability testing, having a good understanding of the utilities or solutions that we use to achieve High Availability in WebSphere Commerce site is very important. For example, if we want to use DB2 High Availability and Disaster Recovery (HADR) to achieve High Availability in the WebSphere Commerce database tier, several concepts should be kept in mind:
- The mechanism of HADR to achieve High Availability in a database
- Which outage scenarios HADR covers
- Whether there are any pre-implementation recommendations from HADR
- Whether there are any limitations in HADR which that impact the implementation
The general approach to apply High Availability testing on a WebSphere Commerce Web site can be classified as:
- Before enabling the High Availability utilities, execute a normal performance test case to establish a baseline for performance comparison purposes. Most times, the normal ratio for performance testing scenarios is not appropriate for High Availability testing, since we should simulate an appropriate and realistic scenario to test the behavior of High Availability utilities. In that case, appropriate customization is needed to implement the High Availability testing.
- Implement the High Availability utilities on the WebSphere Commerce site, and evaluate whether it is functional.
- With High Availability utilities enabled, drive normal performance execution to identify the performance impact to an additional High Availability configuration. We should collect the data for site throughput, error rate, resource (CPU, memory, disk I/O) utilization, and network status. With those dates on hand, compare the results one by one to identify and establish an overall picture of the performance impact of High Availability components enabled on the WebSphere Commerce site.
- Simulate the planned or unplanned outage by automatic script or manually, to identify the performance overhead brought from different outages (or we can say that some of them are disasters), and how can High Availability utilities help the Commerce site to achieve failover.
- With most of the outage scenarios' data collected, draft a performance comparison table to identify how much benefit the candidate High Availability solution gives to the WebSphere Commerce site. This table will help to make an appropriate/reasonable business decision about how to achieve High Availability on the Commerce site.
The key point here is that High Availability is much more a concept built on the customer's point of view, so that the comparison and judgement according to the result that we got should be assessed and evaluated from the customer's view, which sometimes is the most difficult step in driving the performance test.
xxxx